Papers with supervised finetuning
Equipping Language Models with Tool Use Capability for Tabular Data Analysis in Finance (2024.eacl-short)
Copied to clipboard
| Challenge: | Large language models (LLMs) have an array of reasoning capabilities but face limitations such as error propagation and hallucination. |
| Approach: | They propose to use a LLAMA-2 13B CHAT model to act as a task router and task solver to offload certain reasoning steps to external tools that are more suited for the task. |
| Outcome: | The proposed model improves by 35.2% and 5.06% over baseline models and strong GPT-3.5 results. |
WebSTAR: Scalable Data Synthesis for Computer Use Agents with Step-Level Filtering (2026.acl-long)
Copied to clipboard
| Challenge: | Existing datasets rely on human demonstrations, limiting scalability. |
| Approach: | They propose a scalable data synthesis pipeline that transforms noisy rollouts into reliable supervision without human annotation. |
| Outcome: | The proposed pipeline transforms noisy rollouts into reliable supervision without human annotation. |
The olmOCR Project: Building Fully Open OCR using VLMs (2026.acl-demo)
Copied to clipboard
| Challenge: | olmOCR is a fully open OCR system developed through iterative public releases and community feedback. |
| Approach: | They propose an open OCR system that combines a 7B vision-language model trained in two stages: finetuning and reinforcement learning with visual unit tests. |
| Outcome: | The proposed system achieves state-of-the-art performance among open systems and proprietary APIs at a fraction of the cost. |
Reuse Your Rewards: Reward Model Transfer for Zero-Shot Cross-Lingual Alignment (2024.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual human preference data are difficult to obtain at scale, making it challenging to extend this framework to diverse languages. |
| Approach: | They propose a method where a reward model is trained on preference data in one source language and applied to other target languages. |
| Outcome: | The proposed approach is effective under comprehensive evaluation settings, including human evaluation. |
Direct Judgement Preference Optimization (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing judge models are largely trained with supervised finetuning on small data scales to perform limited types of evaluation tasks, limiting generalization. |
| Approach: | They propose to train judge models at large data scales with direct preference optimization . they use four training tasks to form three types of preference pairs targeting different aspects of evaluation . |
| Outcome: | The proposed model outperforms GPT-4o and other similar models on 13 benchmarks. |
Large Language Models Do Multi-Label Classification Differently (2025.emnlp-main)
Copied to clipboard
| Challenge: | Multi-label classification is prevalent in real-world settings, but the behavior of Large Language Models (LLMs) in this setting is understudied. |
| Approach: | They propose to use initial probability distributions to analyze output distributions of LLMs at each label generation step to find out how LLM models perform multi-label classification. |
| Outcome: | The proposed methods improve alignment and predictive performance over existing methods. |
SAND: Boosting LLM Agents with Self-Taught Action Deliberation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Model (LLM) agents finetuned with supervised finetuning may over-commit towards seemingly plausible but suboptimal actions due to limited action space exploration. |
| Approach: | They propose a self-taught actioN deliberation framework that allows LLM agents to explicitly deliberate over candidate actions before committing to one. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on two representative interactive agent tasks and achieves an average 20% improvement over initial finetuning. |
The Language Barrier: Dissecting Safety Challenges of LLMs in Multilingual Contexts (2024.findings-acl)
Copied to clipboard
Lingfeng Shen, Weiting Tan, Sihao Chen, Yunmo Chen, Jingyu Zhang, Haoran Xu, Boyuan Zheng, Philipp Koehn, Daniel Khashabi
| Challenge: | Recent studies show that malicious prompt instructions could solicit objectionable content from LLMs. |
| Approach: | They compare how state-of-the-art LLMs respond to malicious prompts in different languages . they find that LLM's generate unsafe responses more often when a prompt is written in a lower-resource language . |
| Outcome: | The proposed model can generate unsafe responses more often when a malicious prompt is written in a lower-resource language, and less irrelevant responses when written in lower-source languages. |
Concept-skill Transferability-based Data Selection for Large Vision-Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Vision-Language Models (LVLMs) require instruction tuning on extensive data . training on large VL datasets can be prohibitively expensive . |
| Approach: | They propose a data selection technique that uses a small model as a reference model to select training data for efficient finetuning of a target LVLM. |
| Outcome: | The proposed method achieves superior performance and data selection efficiency against 8 strong baselines on two distinct datasets: LLaVA-1.5 and Vision-Flan. |
Distill and Align Decomposition for Enhanced Claim Verification (2026.findings-eacl)
Copied to clipboard
Jabez Magomere, Elena Kochkina, Samuel Mensah, Simerjot Kaur, Fernando Acero, Arturo Oncevay, Charese Smiley, Xiaomo Liu, Manuela Veloso
| Challenge: | Existing methods for complex claim verification struggle to align decomposition quality with verification performance. |
| Approach: | They propose a reinforcement learning approach that optimizes decomposition quality and verifier alignment using Group Relative Policy Optimization. |
| Outcome: | The proposed method outperforms prompt-based approaches and existing methods in six evaluation settings. |
SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationales (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to elicit confidence from large language models are limited to binary or inaccurate group-level confidence estimates. |
| Approach: | They propose a training framework that teaches LLMs to express more fine-grained confidence estimates. |
| Outcome: | The proposed training framework reduces the confidence calibration error and maintains the performance of the model. |
Distilling an End-to-End Voice Assistant Without Instruction Training Data (2025.acl-long)
Copied to clipboard
| Challenge: | Recent efforts to train speech-only LLMs have led to models “forging” speech information from text-only models. |
| Approach: | They propose a paradigm for training Speech Large Language Models without instruction data by using the response of a text-only LLM to transcripts as self-supervision. |
| Outcome: | The proposed model generalizes to Spoken Question Answering, Classification, and Translation and achieves a 72% win rate compared with state-of-the-art models like Qwen 2 Audio . |
Semi-Supervised Reward Modeling via Iterative Self-Training (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Reward models capture values and preferences of humans and are used in Reinforcement Learning with Human Feedback (RLHF) Traditionally, training large language models relies on extensive human-annotated preference data, which poses significant challenges in terms of scalability and cost. |
| Approach: | They propose a method that enhances RM training using unlabeled data. |
| Outcome: | The proposed approach improves reward models without incurring additional labeling costs on unlabeled datasets. |
End-to-End Training of Neural Retrievers for Open-Domain Question Answering (2021.acl-long)
Copied to clipboard
Devendra Sachan, Mostofa Patwary, Mohammad Shoeybi, Neel Kant, Wei Ping, William L. Hamilton, Bryan Catanzaro
| Challenge: | Recent work on training neural retrievers for open-domain question answering (OpenQA) has employed both supervised and unsupervised methods. |
| Approach: | They propose an approach of unsupervised pre-training with the Inverse Cloze Task and masked salient spans followed by supervised finetuning using question-context pairs. |
| Outcome: | The proposed approach outperforms models like REALM and RAG in retrieval accuracy and answer extraction. |
Can GRPO Boost Complex Multimodal Table Understanding? (2025.emnlp-main)
Copied to clipboard
Xiaoqiang Kang, Shengen Wu, Zimu Wang, Yilin Liu, Xiaobo Jin, Kaizhu Huang, Wei Wang, Yutao Yue, Xiaowei Huang, Qiufeng Wang
| Challenge: | Existing table understanding methods struggle with low initialization accuracy and coarse rewards in tabular contexts. |
| Approach: | They propose a three-stage RL framework that enhances multimodal table understanding through: (1) Warm-up that prompts initial perception and reasoning capabilities; (2) Perception Alignment GRPO (PA-GRPO); (3) Hint-Completion GR PO (HC-GRP); |
| Outcome: | The proposed framework outperforms existing models on held-in and held-out datasets, outperforming SFT and GRPO largely. |
Crystal: Introspective Reasoners Reinforced with Self-Feedback (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge-augmented reasoning methods fail to capture the *introspective* nature of knowledge required in commonsense reasoning. |
| Approach: | They propose a method to develop an introspective commonsense reasoner that introspects for knowledge statements related to the given question and makes an informed prediction. |
| Outcome: | The proposed method outperforms standard supervised finetuning and chain-of-thought distilled methods and enhances the transparency of the commonsense reasoning process. |
Robust Data Watermarking in Language Models by Injecting Fictitious Knowledge (2025.findings-acl)
Copied to clipboard
| Challenge: | Data watermarking in language models injects traceable signals, such as specific token sequences or stylistic patterns, into copyrighted text, allowing copyright holders to track and verify training data ownership. |
| Approach: | They propose a data watermarking approach that injects coherent and plausible yet fictitious knowledge into training data using generated passages describing a fictious entity and its associated attributes. |
| Outcome: | The proposed method is designed to be memorized by the LLM, and that increasing their density, length, and diversity of attributes strengthens their memorization. |
Multistage Collaborative Knowledge Distillation from a Large Language Model for Semi-Supervised Sequence Generation (2024.acl-long)
Copied to clipboard
Jiachen Zhao, Wenlong Zhao, Andrew Drozdov, Benjamin Rozonoyer, Md Arafat Sultan, Jay-Yoon Lee, Mohit Iyyer, Andrew McCallum
| Challenge: | Low-resource tasks such as semi-supervised sequence generation require expert knowledge and cost. |
| Approach: | They propose a method for semi-supervised sequence generation where few examples are too scarce to fine tune a model. |
| Outcome: | The proposed method can generalize better than its teacher to unseen examples on semi-supervised sequence generation tasks. |
Retrieval-augmented GUI Agents with Generative Guidelines (2025.emnlp-main)
Copied to clipboard
| Challenge: | GUI agents powered by vision-language models struggle with real-world tasks due to their complex nature and limited training data. |
| Approach: | They propose a lightweight vision-language model that leverages web tutorials at inferencetime to synthesize GUI agents. |
| Outcome: | The proposed agent outperforms baseline GUI agents and surpasses other inference baselines by 2.6% to 13.3% across two model sizes. |
From Tarzan to Tolkien: Controlling the Language Proficiency Level of LLMs for Content Generation (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) often output text at a native level of speech, making them difficult to use for contexts where end-users are not fully proficient. |
| Approach: | They propose a framework to control the difficulty level of text generated by Large Language Models for contexts where end-users are not fully proficient. |
| Outcome: | The proposed framework surpasses GPT-4 and other models at fraction of the cost. |
Improving Task Diversity in Label Efficient Supervised Finetuning of LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable capabilities across domains . but, for challenging tasks, finetuning often requires substantial human annotations - a process that is time-consuming, labor-intensive, and expensive . |
| Approach: | They propose a method that leverages task-diversity as a principle for effective data selection. |
| Outcome: | The proposed method achieves better accuracy than training on the complete dataset (4% increase in MMLU score). |
Rethinking Pragmatics in Large Language Models: Towards Open-Ended Evaluation and Preference Tuning (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to assess social-pragmatic inference in large language models are inadequacy, and preferential tuning is the best approach. |
| Approach: | They propose to use free-form models' responses as a measure to assess social-pragmatic reasoning and advocate for preference optimization over supervised finetuning (SFT). |
| Outcome: | The proposed model outperforms supervised finetuning (SFT) and offers a near-free launch in pragmatic abilities without compromising general capabilities. |
ATLAS: Agent Tuning via Learning Critical Steps (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing agent tuning approaches employ supervised finetuning on entire expert trajectories, but behavior-cloning of full traitories introduces expert bias and weakens generalization to states not covered by the expert data. |
| Approach: | They propose a method that finetunes LLMs on critical steps in expert trajectories and identifies and finetuns them on these steps with reduced costs. |
| Outcome: | The proposed method outperforms existing methods and open-source LLM agents on only 30% critical steps in extensive experiments. |
GSM-Noise: Exploring and Enhancing Large Language Models’ Reasoning under Noisy Inputs (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models struggle when dealing with complex, ill-formed, or noisy inputs . open-source models are less robust, while closed-source ones are more robust . |
| Approach: | They propose to use GSM-Noise to refine inputs before engaging in in-depth analysis to improve LLM robustness under noisy conditions. |
| Outcome: | The proposed model can achieve consistent performance gains under noisy conditions with prompt engineering, supervised finetuning, and reinforcement learning. |